Combining Audio and Video Surveillance with a Mobile Robot

This paper presents a Distributed Perception System for application of intelligent surveillance. The audio and video sensors distributed in the environment are used as a single sensor to reveal and track the presence of a person in the surveilled environment. The system prototype presented in this paper is composed of a static acoustic agent and a static vision agent cooperating with a mobile vision agent mounted on a mobile robot. The mobile robot extends the capabilities of the system by adding a mobile sensor, in this work an omnidirectional camera. The mobile omnidirectional camera can be used to have a closer look of the scene or to inspect portions of the environment not covered by the fix sensory agents. In this paper, the hardware and the software architecture of the system and of its sensors are presented. Experiments on the integration of the audio localization data and on the video localization data are reported.